WHAT IS CLAIMED IS: 

1. A three dimensional modeling apparatus for generating 
three dimensional shape data of a three dimensional object, 

5 comprising: 

projection means for projecting pattern light onto said three 
dimensional object; 

image input means for inputting an image of said three 
dimensional object; 
10 rough shape calculation means for calculating a rough shape 

from said image; 

detailed shape calculation means for calculating a detailed 
shape from said image; and 

three dimensional shape data generating means for generating 
15 three dimensional shape data of said three dimensional object based 
on said rough shape and said detailed shape, 

2. A three dimensional modeling apparatus for generating 
three dimensional shape data of a three dimensional object, 

20 comprising: 

projection means for projecting pattern light onto said three 
dimensional object; 

image input means for inputting an image of said three 
dimensional object; 
25 rough shape input means for inputting a rough shape of said 

three dimensional object; 

detailed shape calculation means for calculating a detailed 
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shape from said image; and 

three dimensional shape data generating means for generating 
three dimensional shape data of said three dimensional object based 
on said rough shape and said detailed shape. 

5 

3. A three dimensional modeling apparatus for generating 
three dimensional shape data of a three dimensional object, 
comprising : 

image input means for inputting an image of said three 
10 dimensional object, said image being obtained by projecting 
pattern light onto said three dimensional object; 

rough shape calculation means for calculating a rough shape 
from said image; 

detailed shape calculation means for calculating a detailed 
15 shape from said image; and 

three dimensional shape data generating means for generating 
three dimensional shape data of said three dimensional object based 
on said rough shape and said detailed shape. 

20 4. A three dimensional modeling apparatus for generating 

three dimensional shape data of a three dimensional object, 
comprising: 

image input means for inputting an image of said three 
dimensional object, said image being obtained by projecting 
25 pattern light onto said three dimensional object; 

rough shape input means for inputting a rough shape of said 
three dimensional object; 
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detailed shape calculation means for calculating a detailed 
shape from said image; and 

three dimensional shape data generating means for generating 
three dimensional shape data of said three dimensional object based 
5 on said rough shape and said detailed shape. 

5 . A three dimensional modeling apparatus according to claim 

1, 

wherein said three dimensional shape data generating means 
10 determines the final three dimensional shape data based on the 
following rule; if the detailed shape exits inside the rough shape , 
then the detailed shape is the final three dimensional shape data 
of said object, otherwise the rough shape is taken as the final 
three dimensional shape data of said object, 

15 

6 . A three dimensional modeling apparatus according to claim 

1, 

wherein said three dimensional shape data generating means 
obtains a region where the object surface can exist based on the 
20 rough shape, to generate three dimensional shape data of said three 
dimensional object based on the rough shape, the detailed shape, 
and the region where the object surface can exist . 

7 . A three dimensional modeling apparatus according to claim 

25 1, 

wherein said detailed shape calculation means obtains a 
plurality of object surface location candidates from the input 
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image . 

8 . A three dimensional modeling apparatus according to claim 

If 

5 wherein said detailed shape calculation means sets a region 

where the object surface cannot exist in a portion inside the rough 
shape and determines a region inside the rough shape other than 
said region where the object surface cannot exist as a region where 
the object surface can exist, to generate three dimensional shape 
10 data of said three dimensional object based on the rough shape, 
the detailed shape, and the region where the object surface can 
exist . 

9 . A three dimensional modeling apparatus according to claim 

15 1, 

wherein said three dimensional shape data generating means 
determines a final object surface location candidate from a 
plurality of object surface location candidates based on a region 
where the object surface can exist and generates three dimensional 
20 shape data of said three dimensional object based on said final 
object surface location candidate. 

10. A three dimensional modeling apparatus according to 
claim 1, 

25 wherein the rough shape is calculated by said rough shape 

calculation means based on silhouette information obtained from 
the object image input by said image input means. 
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11, A three dimensional shape data recording apparatus for 
recording three dimensional shape data of a three dimensional 
ob j ect , comprising : 

three dimensional shape data generation means for generating 
5 three dimensional shape data of said three dimensional object; 

three dimensional shape data reliability assigning means for 
assigning a data reliability evaluation value to each of said three 
dimensional shape data; and 

three dimensional shape data recording means for recording 
10 three dimensional shape data to which said reliability evaluation 
value is assigned in a medium. 

12 . A three dimensional shape data recording apparatus for 
recording three dimensional shape data of a three dimensional 

15 object, comprising: 

three dimensional shape data capturing means for capturing 
three dimensional shape data of said three dimensional object; 

three dimensional shape data reliability assigning means for 
assigning a data reliability evaluation value to each of said three 
20 dimensional shape data; and 

three dimensional shape data recording means for recording 
three dimensional shape data to which said reliability evaluation 
value is assigned in a medium. 

25 13 * A three dimensional modeling method for generating three 

dimensional shape data of a three dimensional object, comprising 
the steps of : 
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projecting pattern light onto said three dimensional object; 
inputting an image of said three dimensional object; 
calculating a rough shape from said image; 
calculating a detailed shape from said image; and 
5 generating three dimensional shape data of said three 

dimensional object based on said rough shape and said detailed 
shape. 



14 . A three dimensional modeling method for generating three 
10 dimensional shape data of a three dimensional object, comprising 
the steps of: 

projecting pattern light onto said three dimensional object; 

inputting an image of said three dimensional object; 

capturing a rough shape of said three dimensional object; 
15 calculating a detailed shape from said image; and 

generating three dimensional shape data of said three 
dimensional object based on said rough shape and said detailed 
shape. 

20 15 • A three dimensional modeling method for generating three 

dimensional shape data of a three dimensional object, comprising 
the steps of: 

inputting an image of said three dimensional object, said 
image being obtained by projecting pattern light onto said three 
25 dimensional object; 

calculating a rough shape from said image; 
calculating a detailed shape from said image; and 
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generating three dimensional shape data of said three 
dimensional object based on said rough shape and said detailed 
shape . 

16 . A three dimensional modeling method for generating three 
dimensional shape data of a three dimensional object, comprising 
the steps of: 

inputting an image of said three dimensional object, said 
image being obtained by projecting pattern light onto said three 
dimensional object; 

capturing a rough shape of said three dimensional object; 

calculating a detailed shape from said image; and 

generating three dimensional shape data of said three 
dimensional object based on said rough shape and said detailed 
shape . 

17. A three dimensional modeling method according to claim 

13, 

wherein said step of generating three dimensional shape data 
includes determining the final three dimensional shape data based 
on the following rule; if the detailed shape exits inside the rough 
shape, then the detailed shape is the final three dimensional shape 
data of said object, otherwise the rough shape is taken as the final 
three dimensional shape data of said object. 

18. A three dimensional modeling method according to claim 

13, 
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wherein said step of generating three dimensional shape data 
includes obtaining a region where the object surface can exist 
based on the rough shape, to generate three dimensional shape data 
of said three dimensional object based on the rough shape, the 
5 detailed shape, and the region where the object surface can exist. 

19. A three dimensional modeling method according to claim 

13, 

wherein said step of calculating detailed shape includes 
qIO obtaining a plurality of object surface location candidates from 
|5 the input image. 

i2 20. A three dimensional modeling method according to claim 

r 13 * 

JJS15 wherein said step of calculating detailed shape includes 

r~J setting a region where the object surface cannot exist in a portion 
r? inside the rough shape and determining a region inside the rough 

shape other than said region where the object surface cannot exist 

as a region where the object surface can exist, to generate three 
20 dimensional shape data of said three dimensional object based on 

the rough shape, the detailed shape, and the region where the object 

surface can exist. 

21. A three dimensional modeling method according to claim 

25 13, 

wherein said step of generating three dimensional shape data 
includes determining a final object surface location candidate 
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from a plurality of object surface location candidates based on 
a region where the object surface can exist and generating three 
dimensional shape data of said three dimensional object based on 
said final object surface location candidate. 

5 

22. A three dimensional modeling method according to claim 

13, 

wherein the rough shape is calculated by said rough shape 
calculation means based on silhouette information obtained from 
10 the object image input by said step of inputting an image. 

23. A three dimensional shape data recording method for 
recording three dimensional shape data of a three dimensional 
object, comprising the steps of: 

15 generating three dimensional shape data of said three 

dimensional object; 

assigning a data reliability evaluation value to each of said 
three dimensional shape data; and 

recording three dimensional shape data to which said 
20 reliability evaluation value is assigned in a medium. 

24. A three dimensional shape data recording method for 
recording three dimensional shape data of a three dimensional 
object, comprising the steps of: 

25 capturing three dimensional shape data of said three 

dimensional object; 

assigning a data reliability evaluation value to each of said 
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three dimensional shape data; and 

recording three dimensional shape data to which said 
reliability evaluation value is assigned in a medium. 

5 25. A medium having a program recorded therein for causing 

a computer to generate three dimensional shape data of a three 
dimensional object, said program comprising the steps of: 

controlling pattern light projection section connected to 
said computer for projecting pattern light onto said three 
10 dimensional object; 

inputting an image of said three dimensional object; 
calculating a rough shape from said image; 
calculating a detailed shape from said image; and 
generating three dimensional shape data of said three 
15 dimensional object based on said rough shape and said detailed 
shape . 

26. A medium having a program recorded therein for causing 
a computer to generate three dimensional shape data of a three 
20 dimensional object, said program comprising the steps of: 

controlling a pattern light projection section connected to 
said computer for projecting pattern light onto said three 
dimensional object; 

inputting an image of said three dimensional object; 
25 capturing a rough shape of said three dimensional object; 

calculating a detailed shape from said image; and 

generating three dimensional shape data of said three 
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dimensional object based on said rough shape and said detailed 
shape. 

27. A medium having a program recorded therein for causing 
5 a computer to generate three dimensional shape data of a three 

dimensional object, said program comprising the steps of: 

inputting an image of said three dimensional object, which 
is obtained by projecting pattern light onto said three dimensional 
object; 

10 calculating a rough shape from said image; 

calculating a detailed shape from said image; and 
generating three dimensional shape data of said three 
dimensional object based on said rough shape and said detailed 
shape . 

15 

28. A medium having a program recorded therein for causing 
a computer to generate three dimensional shape data of a three 
dimensional object, said program comprising the steps of: 

inputting an image of said three dimensional object, which 
20 is obtained by projecting pattern light onto said three dimensional 
object; 

capturing a rough shape of said three dimensional object; 
calculating a detailed shape from said image; and 
generating three dimensional shape data of said three 
25 dimensional object based on said rough shape and said detailed 
shape . 
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29. A medium having a three dimensional modeling program 
recorded therein according to claim 25, 

wherein said step of generating three dimensional shape data 
includes determining the final three dimensional shape data based 
5 on the following rule; if the detailed shape exists inside the rough 
shape, then the detailed shape is the final three dimensional shape 
data of said object, otherwise the rough shape is taken as the final 
three dimensional shape data of said object. 

10 30. A medium having a three dimensional modeling program 

recorded therein according to claim 25, 

wherein said step of generating three dimensional shape data 

includes obtaining a region where the object surface can exist 

based on the rough shape, to generate three dimensional shape data 
15 of said three dimensional object based on the rough shape, the 

detailed shape, and the region where the object surface can exist. 

31. A medium having a three dimensional modeling program 
recorded therein according to claim 25, 

20 wherein said step of calculating a detailed shape includes 

obtaining a plurality of object surface location candidates from 
the input image. 

32. A medium having a three dimensional modeling program 
25 recorded therein according to claim 25, 

wherein said step of calculating a detailed shape includes 
setting a region where the object surface cannot exist in a portion 
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inside the rough shape and determining a region inside the rough 
shape other than said region where the object surface cannot exist 
as a region where the object surface can exist, to generate three 
dimensional shape data of said three dimensional object based on 
5 the rough shape, the detailed shape, and the region where the object 
surface can exist* 

33. A medium having a three dimensional modeling program 
recorded therein according to claim 25, 

10 wherein said step of generating three dimensional shape data 

includes determining a final object surface location candidate 
from a plurality of object surface location candidates based on 
a region where the object surface can exist and generating three 
dimensional shape data of said three dimensional object based on 

15 said final object surface location candidate, 

34. A medium having a three dimensional modeling program 
recorded therein according to claim 25, 

wherein the rough shape is calculated by said step of 
20 calculating a rough shape based on silhouette information obtained 
from the object image input by said step of inputting an image. 

35. A medium having a program recorded therein for causing 
a computer to record three dimensional shape data of a three 

25 dimensional object in the medium, said program comprising the steps 
of: 

generating three dimensional shape data of said three 
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dimensional object; 

assigning a data reliability evaluation value to each of said 
three dimensional shape data; and 

recording three dimensional shape data to which said 
5 reliability evaluation value is assigned in the medium. 

36. A medium having a program recorded therein for causing 
a computer to record three dimensional shape data of a three 
dimensional object in the medium, said program comprising the steps 

10 of: 

capturing three dimensional shape data of said three 
dimensional object; 

assigning a data reliability evaluation value to each of said 
three dimensional shape data; and 
15 recording three dimensional shape data to which said 

reliability evaluation value is assigned in the medium. 

37. A medium having three dimensional shape data of a three 
dimensional object recorded therein, said medium holding data 

20 reliability evaluation value information for each of said three 
dimensional shape data. 

38. A three dimensional modeling apparatus for generating 
three dimensional shape data of a three dimensional object, 

25 comprising: 

projection means for projecting pattern light onto said three 
dimensional object; 
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image input means for inputting an image of said three 
dimensional object; and 

three dimensional shape calculation means for calculating a 
three dimensional shape of said three dimensional object from the 
5 input image, 

wherein said three dimensional shape calculation means 
extracts a pattern projected region and a pattern border region 
in said input image to calculate the three dimensional shape of 
said three dimensional object based on these regions. 

10 

% 39. A three dimensional modeling apparatus according to 

% claim 1, 

2! wherein said detailed shape calculation means extracts a 

f" pattern projected region and a pattern border region in said input 
ML 5 image to calculate the detailed shape of said three dimensional 
object based on these regions. 

40. A three dimensional modeling apparatus according to 
claim 38, 

20 wherein said pattern light has a plurality of binary patterns . 

41. A three dimensional modeling apparatus according to 
claim 38, 

wherein said image input means performs image input from at 
25 least two positions. 

42. A three dimensional modeling apparatus for generating 
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three dimensional shape data of a three dimensional object, 
comprising: 

projection means for projecting pattern light onto said three 
dimensional object; 

image input means for inputting an image of said three 
dimensional object; and 

three dimensional shape calculation means for calculating a 
three dimensional shape of said three dimensional object from the 
input image, 

wherein said image input means performs image input from at 
least two positions, 

said three dimensional shape calculation means extracts a 
portion in said input image where color or density changes, and 

matching of portions where color or density changes is 
performed between input images, to calculate the three dimensional 
shape of said three dimensional object. 

43. A three dimensional modeling apparatus according to 
claim 1, 

wherein said image input means performs image input from at 
least two points at different locations, 

said detailed shape calculation means extracts a portion in 
said input image where color or density changes, and 

matching of portions where color or density changes is 
performed between input images, to calculate the three dimensional 
shape of said three dimensional object. 
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44. A three dimensional modeling apparatus according to 
claim 3, 

wherein said image input means performs image input where the 
images are captured at least two different points at different 
5 locations, 

said detailed shape calculation means extracts a portion in 
said input image where color or density changes, and 

matching of potions where color or density changes is 
performed between input images, to calculate the three dimensional 
10 shape of said three dimensional object. 

HF 45. A three dimensional modeling apparatus according to 

O claim 42, 

M* wherein said pattern light for projection includes multi- 

CES color patterns in which adjacent patterns have different colors 
O having a hue difference of at least 90 degree or more, or a 
□ brightness difference of 0.3 or more, when the colors are 
represented in an HSV space. 

20 46. A three dimensional modeling apparatus according to 

claim 42, 

wherein said matching of the portions where color or density 
changes is obtained based on properties of the portions where color 
or density changes, which are obtained from the input images. 

25 

47. A three dimensional modeling apparatus according to 
claim 46, 
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wherein the property of the portion where color or density 
changes is color information regarding portions located to the left 
and right of, or above and below, said portion where color or density 
changes in said input image. 

5 

48. A three dimensional modeling apparatus according to 
claim 46, 

wherein averaging is performed on the input images when said 
property is obtained. 

10 

49. A three dimensional modeling apparatus according to 
claim 48, 

wherein said averaging is performed for each of divided 
regions of the input image. 

15 

50. A three dimensional modeling apparatus according to 
claim 1 , further comprising moving means for moving said projection 
means and said image input means. 

20 51. A three dimensional modeling apparatus according to 

claim 50, 

wherein said moving means moves said projection means and said 
image input means based on said rough shape. 

25 52. A three dimensional modeling apparatus for generating 

three dimensional shape data of a three dimensional object, 
comprising: 
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at least two three dimensional shape input means for obtaining 
a three dimensional shape of said three dimensional object; and 

moving means for moving said three dimensional shape input 
means , 

5 wherein said three dimensional shape input means have 

different three dimensional input accuracy* 

53. A three dimensional modeling apparatus according to 
claim 52 , 

10 wherein said moving means moves said three dimensional shape 

input means having higher three dimensional input accuracy based 
on three dimensional shape information obtained by said three 
dimensional shape input means having lower three dimensional input 
accuracy. 

15 

54 . A three dimensional modeling method for generating three 
dimensional shape data of a three dimensional object, comprising 
the steps of: 

projecting pattern light onto said three dimensional object; 
20 inputting an image of said three dimensional object; and 

calculating a three dimensional shape of said three 
dimensional object from the input image, 

wherein said step of calculating the three dimensional shape 
includes extracting a pattern projected region and a pattern border 
25 region in said input image to calculate the three dimensional shape 
of said three dimensional object based on these regions. 
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55. A three dimensional modeling method according to claim 

13, 

wherein said step of calculating the detailed shape includes 
extracting a pattern projected region and a pattern border region 
5 in said input image to calculate the detailed shape of said three 
dimensional object based on these regions. 

56. A three dimensional modeling method according to claim 

54, 

10 wherein said pattern light has a plurality of binary patterns * 

57. A three dimensional modeling method according to claim 

54, 

wherein said step of inputting an image includes performing 
15 image input from at least two positions. 

58 . A three dimensional modeling method for generating three 
dimensional shape data of a three dimensional object, comprising 
the steps of : 

20 projecting pattern light onto said three dimensional object; 

inputting an image of said three dimensional object; and 
calculating a three dimensional shape of said three 
dimensional object from the input image, 

wherein said step of inputting an image includes performing 
25 image input from at least two positions, 

said step of calculating the three dimensional shape includes 
extracting a portion in said input image where color or density 
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changes , and 

matching portions where color or density changes between 
input images , to calculate the three dimensional shape of said 
three dimensional object. 

5 

59. A three dimensional modeling method according to claim 

13, 

wherein said step of inputting an image includes performing 
image input where the images are captured at least two points at 
10 different locations, 

said step of calculating a detailed shape includes extracting 
a portion in said input image where color or density change, and 
matching portions where color or density changes between 
input images, to calculate the three dimensional shape of said 
15 three dimensional object. 

60. A three dimensional modeling method according to claim 

58, 

wherein said pattern light for projection includes multi- 
20 color patterns in which adjacent patterns have different colors 
having a hue difference of at least 90 degrees or more, or a 
brightness difference of 0.3 or more, when the colors are 
represented in an HSV space. 

25 61. A three dimensional modeling method according to claim 

58, 

wherein said matching of the portions where color or density 
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changes is obtained based on properties of the portions where color 
or density changes, which are obtained from the input image. 

62. A three dimensional modeling method according to claim 

5 61, 

wherein the property of the portion where color or density 
changes is color information regarding portions located to the left 
and right of , or above and below, said portion where color or density 
changes in said input images. 

10 

63. A three dimensional modeling method according to claim 

61, 

wherein averaging is performed on the input images when said 
property is obtained . 

15 

64. A three dimensional modeling method according to claim 

63, 

wherein said averaging is performed for each of divided 
regions of the input images. 

20 

65 . A three dimensional modeling method for generating three 
dimensional shape data of a three dimensional object, comprising: 

at least two three-dimensional shape inputting steps of 
obtaining the three dimensional shape of said three dimensional 

25 object; and 

a step of moving a three dimensional shape input section; 
wherein said three dimensional shape inputting steps have 
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different three dimensional input accuracy. 

66* A three dimensional modeling apparatus for generating 
three dimensional shape data of a three dimensional object, 
5 comprising : 

first three dimensional shape measuring means for obtaining 
rough shape information of said three dimensional object; 

second three dimensional shape measuring means for obtaining 
detailed shape information of said three dimensional object; and 
10 a three dimensional shape data generating section for 

generating the three dimensional shape data of said three 
dimensional object based on said rough shape information and said 
detailed shape information . 

15 67 . A three dimensional modeling apparatus according to 

claim 66 , further comprising control means for controlling the 
position of said second three dimensional shape measuring means, 
said control means being controlled based on said rough shape 
information obtained by said first three dimensional measuring 

20 means . 

68 . A three dimensional modeling method for generating three 
dimensional shape data of a three dimensional object, comprising: 

a first three dimensional shape measuring step of obtaining 
25 rough shape information of said three dimensional object; 

a second three dimensional shape measuring step of obtaining 
detailed shape information of said three dimensional object; and 
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a three dimensional shape data generating step of generating 
the three dimensional shape data of said three dimensional object 
based on said rough shape information and said detailed shape 
information. 

5 

69. A medium having a program recorded therein for causing 
a computer to generate three dimensional shape data of a three 
dimensional object, said program comprising: 

a first three dimensional shape measuring step of obtaining 
10 rough shape information of said three dimensional object; 

a second three dimensional shape measuring step of obtaining 
detailed shape information of said three dimensional object; and 

a three dimensional shape data generating step of generating 
the three dimensional shape data of said three dimensional object 
15 based on said rough shape information and said detailed shape 
information. 
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